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A series of workshops were offered for vocational 
education administrators and researchers to train them in 
institutional research methods. in a systems planning context. The 
basic purpose of the project was to provide an opportunity for 
vocational educators to become familiar with (1) research 
methodology; (2) to train vocational educators to utilize research 
services which now exist on their own campuses; and (3) to develop 
institutional research skills in personnel on campuses where such 
competency does not exist. The hypothesis of the workshop was: 
experiencing first hand and applying the research concepts to 
planning and evaluation problems will increase the likelihood of 
vocational educators using such services on his own campus. 
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INTRODUCTION 



This is a final report on the series of research training workshops 
offerexd by the Nor Cal Research Group and sponsored by the Division of 
Occupational Education of the California Community College Chancellor's 
Office, The intent of these workshops was to make vocational adminis- 
trators more aware of institutional research methods and to acquaint 
them with research and planning resources available to them. 

The report begins with a narrative description of the workshops by 
the project auditor and is followed by materials which essentially de-- 
scribe each of the three workshops* The report does not contain a de- 
scription of the individual projects which arose on college campuses in 
conjunction with the workshops. It was agreed in the project application 
that a year would be allowed from the termination of the project to as- 
sess the learning which took place in the workshops. As of this writing, 
eight of the twenty-three colleges participating in the workshops have 
substantially ccmpleted research projects undertaken in conjunction with 

the research training workshops* The purpose of these workshops was not, 

/ / 

however, limited to the dev'eiopment of a single, formally constructed 
research project by the workshop participants. It was rather to provide 
practical research skills to improve the day-to-day Institutional re- 
search requirements of community colleges, such as student follow up, 
program feasibility studies, and curriculum evaluation. 

On behalf of the Nor Cal Research Group, 1 would like to thank Dr. 
Bill Morris, Consultant on Evaluation of Vocational Education with the 
C'»lifornia Community Colleges Chancellor's Office, for his cooperation 



2 



and participation in these workshops- I would also like to commend the 
workshop coordinators, Lorine Aughinbaugh, Assistant Dean of Research, 
Merican River College; Dean Eaton, Director of Admissions, Solano Cbl- 
.lege; Tom MacMillan, Director of Institutional Research, Santa Barbara 
Gity College, and Paul Preising, Directosr of Institutional Research, 
Sari Jose City College, for arranging facilities on community college cam- 
puses and for their thorough preparation in vihelr roles as group discus- 
sion leaders and resource persons. 

I would also like to commend Dr. Ben Gold, Director of Research, 
Los Angeles City College, who acted as the chief project consultant, 
for his invaluable assistance in both planning workshop activities 
and offering leadership in carrying them to successful conclusion. The 
general feeling cf workshop participants was that Dr^ Arthur Cherdack, 
Director of Research, East Los Angeles College; Dr. Ralph Smith, Chairman, 
Department of Educational Administration, Brigham Young University, and 
Dr. Kent Stephens, Department of Educational Administration, Brigham 
Young University, did an effective job in presenting educational system 
analysis concepts and techrdques« 

I would also like to thank Ira Neiken, project auditor, for his fair 
and comprehensive educational audit of the project. 
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I. ABSTRACT 
OF NORTHERN CALTFOrxNTA TRAINING WORKSHOP FOR 
VOCATIONAL EDUCATORS 

A series of workshops were offered for vocational education administrators 
and researchers to train them in institutional research methods in a 
systems planning context. This resulted from awareness that vocational 
education administrators were not aware of the research skills available 
to them in the research office on their own campus nor were the researchers 
adequately prepared to deal with administrative planning encountered in the 
development of vocational education programs. The workshops were staffed 
by leading cotisultants in systems planning and community college institu- 
tional research directors. The problem areas identified for investigation 
during thfi workshops were cast in appropriate systems planning models and 
teams began to carry out the necessary research for problem solution. 

The basic purpose of the project is to provide an opportunity for vocational 
educators to become familiar with (1) research methodolQgy; (2) to train 
vocational educators to utilize research services which now exist on their 
own campuses; and (3) to develop institutional research skills in personnel 
on campuses where such competency does not exist« The hypothesis of the 
workshop was: experiencing first hand and applying the research concepts to 
planning and evaluation problems will increase the likelihood of vocational 
educators using such services on his own campus. 

Too often decisions are made on partial or unreliable information, even 
though the capability for obtaining accurate information may be available. 
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Planning models will only be implemented successfully when thev are tied in 
more general research methodplogy which is essential to their effectiveness. 

*TT. OBJECTIVES OF THE PROJECT 
The project has three objectives: 

1. "To assist teams of two educators from 16 northern California community 
colleges to identify at least one problem area in an important vocational 
education function, to describe the problem in systems terms and to 
conduct research necessary to the problem solution". 

This objectives consists of five measurable portions: 

a. Two people 

b. Fifteen Community Colleges 



c. One problem need to identify 




d. One problem described^ each 

e. Research conducted to solve the problem 

2. "To provide each Community College team specific training in a general 
systems planning model of tlieir choice and to relate this planning model 
to research techniques necessary to the chosen system". 

This objective consists of two portions: 

a. Training in general systems planning models 

b. Relating research techniques to this chosen system 

The Project Director with concurrence from workshop coordinators and 
the Project Auditor believed one generalized generic model could be 
used and modified by all groups at the workshop and therefore 
training in one generalized planning model (a modification of Kent 
Stephen model) was used. 

ERJC 
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3» "To Increase the probability that the vocational educator from partici- 
pating colleges will use the research resources available on college 
campus". 

This objectives consists of one part: increasing the probability of 
vocational educators using resources available from campus. This objective 
is the most difficult to measure of the three objectives of the project • 

III. EVALUATION OF OBJECTIVES 

Objective //I was measured by: (a) attendance and participation of vocational 
personnel; (b) selection and application of assistance design to an actual 
program; and (c) a survey of the vocational educators regarding the assistance 
^ - given during the workshops and the understanding of- the various planning 

models presented. Measurement of objective #1 was competently and adequately 
accomplished. 

Objective #2 was measured by: (a) actual changes made or planned in the vocational 
programs selected for study as a result of information gained during analysis 
of ::heir data; (b) a presentation by conference participants on their projects 
during the final workshop rated by the Project Auditor on (1) understar^ing 
of the planning model, (2) application of the model to a particular Planning 
or revision problem, and (3) appropriateness of the research design and 
statistical methods used. 

Objective //3 was to be measured by standardized interviews of vocational 
administrators who participated prior to the workshop to determine (a) the 
number of administrators currently using the systems planning models and the 
extent to which research resources on campus are currently used and (b) one 

ERIC 



'year after the coinpletion of the workshop surveys to particpacing vocational 
administrators to' determine how many continue to use information and skills 
on other projects; €»g. (1) the local team approach, (2) the planning model 
used. It was decided by the Project Director (the Auditor concurs) that (a) 
was not necessary. The important information is contained in the discrepancy 
between what is being used now ari^^hat will be used in the future and this 
can best be measured by a standardized follow-up of the administrators after 
the completion of the workshops to determine how many continue to use 
information gained during the project. The post measurement (b) was deter- 
mined by the Auditor to be extremely adequate and a highly competent measure- 
ment/of the actual effects of the workshop. The difficulty is this can not 
be done for a year. The Auditor is confident the present Project Director 
will undertake this post eyalua-tion of the workshop series and will be highly 
competent and successful in reporting the results of the follow-up survey 
conducted. 

The only objectivie which lends itself easily to statistical analysis is 
objective //3, where the significance of the change in the variables measured 
can be determined. The results for this objective will no^: be available for 
another year, v 

IV. ACTIVITIES TO MEET OBJECTIVES 

An advisory committee representing both vocational educators and the Nor-Cal 
Steering Committee met and planned three workshops which were conducted on 
November 3-4, 1972, December 7-8, 1972 and March 22-23, 1973* Vocational, 
educators and research personnel from all the northern California Community 

Colleges were invited. A director was selected for each workshop and 

consultants were used as workshop staff. 
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At Workshop 1 widely used planning models were presented by experts in the 
field with special attention given to operational strengths and weaknesses 
of each. Each participating team selected a design methodology appropriate 
to its problem following the presentations, reviewed and selected the program 
characteristics appropriate to the model selected and to their local require- 
ments and developed a tentative time schedmle and a list of activities to be 
completed by the team, Ben Gold was the outside consultant for the research 
methodology and design. Kent Stephen^s desi<2;n for systems analysis and - 
problems was used. Staff from the various Nor-Cal Community Colleges, especially 
Lorine Aughinbaugh, Tom McMillan and Paul Preising aided in workshop 
development and implementation. 

Workshop 2 was used to review progress to date with the aid of the consultants, 
to review and revise team operational plans, to finalize plans for completion 
of each study and to allow for progress reports and plan completion of the 
study by each participating team to the entire group. 

Workshop 3 was used as (1) a presentation of final progress reports projects, 
(2) a dissemination of exemplary developed follow*-up and needs assessment 
practices, (3) an evaluation of planning models and the use of a planning - 
model for the situation for which it was selected, and (4) the evaluation 
of the workshop series In its entirety and preparation and dissemination of 
final project report. 

Memoranda for each workshcJp to Nor-Cal research training workshop potential 
participants and for participants were sent out from the Project Director's 
office prior to each meeting, outlining specifically the tentative content 
of the workshop and the potentials which the workshop could and was to serve. 
\ comprehensive attempt was made to involve oarticipants arid potential. 
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participants in workshops and in pre-workshop activities whenever possible. 
The Nor-Cal Steering Advisory Comnitrtee met several times to analyze and 
plan and evaluate workshops • 

An addendum was produced to clarify the nature of the research training con- 
ducted at the project v/orkshops. The research training tnet two criteria: 
(1) all research methods and techniques covered in the workshops were elementary 
enough to be understood and applied in the two days allotted for each workshop, 
and (2) all methods and techniques were immediately applicable to research 
problems commonly encountered in the community college setting- 

A generic system model by Dr. Roger Kaufman, desbirbed in a vocational educa- 
' tion seminar in San Diego, was adopted to underpin the essential research 
competencies needed in two key areas: (1) problem identif ic.ation, and (2) 
determining performance effectiveness* 

Several sections of the workshop related to identifying problems from needs. 
Needs assessment ina system approach assumes certain methods, the availability 
of certain methods, and an understanding of these methods by the vocational 
educator and research director. Sections on the use of 1970 census data as a 
means of pinpointing population characteristics, the Delphi technique as a 
tool in needs assessment, and the construction of questionnaires, rating 
scales, and standardized interviews for a given target population were 
' developed and available at the second workshop. 

To meet the needs of de^termining performance effectiveness, a general 
understanding of the experimental method: hypothesizing, making operational 

ERIC 
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definitions, the nature cf intervening variables, independent and dependent 
variables, and controlled observation was made available. Five widely used 
research designs were advanced and available at the workshops: the one-shot 
design, the one group pre-post test design, the static group design, the 
random group design, and the pre-post randonized group design. Fundamental 
statistical concepts: significance tests, sampling and procedures, grouping 
data, and test scores were available and covered at the workshops. 

V. WORKSHOPS 

The first workshop was opened wita a brief orientation to the total project 
and then assisted workshop participants in delineating the research to be 
undertaken during the year. Workshop participants were introduced to an 
educational system manag<?ment model and operations research concepts and 
presented research designs most appropriate to institutional research in 
the comnunity college. The workshop met with limited success. One of the 
major reasons was the fact that each of the teams at the workshop was at a 
different level of knowledge and training and this Auditor feels that the 
workshop required more individualization than had first been planned. 

The auditor conmends the Project Director and his consultant staff for 
recognizing the problen'is of the first workshop and producing an active 
second workshop which better met the needs of the participants. Each work- 
shop consultant was available at a different table to give assistance to 
participants on their (Particular project and to discuss particular topics 
in specific terms. These topics were: problem definition, experimental 
design, census data analysis, questionnaires and interviews, statistics 
and sampling, and follow-up. This Individualized approach in the second^' 
workshop met with an extremely good reception by the participants and the 




Ludltor noted the success of this second workshop. Actual projects had not 
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to this point been well delineated except in a few cases and therefore limited 
critiquing of projects took place* 

A number of infoi lonal documents had been developed by workshop leaders 
and consultants and were distributed at the workshops and made available to 
all tUo;:;e that wanted such information. The Project Director had devised 
an evaluation sheet for the first workshop, asking comments from uarticioants 
upon each of the presentations, noting ideas which had immediate application 
to the participants* area of responsibility, ideas and concepts the parti- 
cipant would like to hear more about and other comments positive or negative 
that needed to be made about the presentations « The evaluation sheets were 
collected from participants and were used to aid in determining the process 
and content structure of the second workshop. The Auditor commends the 
Project Director upon such a form of process evaluation which he believes 
aided tremendously in the success of the second workshop. 

The Project Director collected and analyzed project descriptions from the 
various teams between the second and thrid workshops. This data can be 
Invaluable for determining the affect of the workshop upon the participants. 

The third workshop emphasized follow-up procedures » needs assessment and 
project progress reports. Performance requirements for a follow-up instrument, 
exemplary follow-up practices/procedures (3 cases) and small group meetings 
to develop a follow-up instrument led to a generalized proto-type follow-up 
instrument which was applicable to met the needs of all the Nor-Cal Coimnunity 
Colleges. The workshop x^as well attended and is considered as successful by 
this auditor* 
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VI, THE AUDITOR 

The Auditor has met on five separate occasions with the Projecr: Director 
and discussed the project in terms of the Auditor's perceptions of a general 
discussion of the project, things that could and couldn't be done, how the 
process was perceived; the project's strengths, weaknesses, reconmenda tions , 
and suggestions. The Auditor also attended the second project workshop and 
the third project workshop and at these workshops met and talked to the 
project's consultants and workshop coordinators, as well as a number of the 
project participants, ^He received valuable information on how those people 
perceived the project and what was happening. The Project Director made 
available to the Project Auditor all written materials with reference to the 
workshop and has been extremely helpful in producing records for the Project 
Auditor to examine. To add depth to the audit, the Auditor chose at random 
two particular teams from two community colleges attending the workshop 
series: one college in the Sierras and the other in the Sacramento Valley 
and has been corresponding and. conversing with these two teams. 

VII. PROJECT STRENGTHS, WEAKNESSES, RECOMMENDATIONS AND SUMMARY 

Project Strenj^ths 

1, Planning for the project has been continuous and process revision has 

been undertaken continuously when necessary* 
2» Workshops have been modified when necessary to meet the needs of the 

participants. . ^ 

3. The project and the workshops have essentially followed the original 
project design and all modifications wb.ich have occurred have been 
concurred upon by the Project Director, consultants, and Project Auditor* 

4, The project is to be commended for the availability of experts to meet 
the needs of the participants. A number of exceptionally well qualified 
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people who are able to Interact effectively with community college 
administrators and researchers were available and used. 

5- The pre/post planning and analysis f or^, the workshops was very well done* 
The Auditor commends the Project Director. 

6» The openness and frankness of the Project Director and coordinators in 
noting the project*s limitations as well as strengths is seen by this 
Auditor as a strength. This has allowed for necessary modifications 
of the project to better meet the participants needs. 

7. Interim process evaluation allowing for revision and modification for 
workshops after original planning had been done produced more successful 
workshops. The Auditor sees the continuous interim process evaluation 
of the project as a significant strength of the project team and the 
project itself. 

8. The continuity of attendance among many of the participants from first 
to second to third workshop is seen as a strength of the project. The 
project was able to hold its participants. 

9. There was excellent one-to-one interaction at the second workshop. 
Participants could go to whom they needed to go to get the information 
or help that they needed. The six-table design was very effective. 

10. Group teams of participants did identify problem areas to work on. 

Project Weaknesses 

1« Some participants and consultants were not sure about the purpose of the 
project. Greater clarification of project objectives was necessary. 

2. The Auditor has determined that a minimum of interaction between personnel 
(participants and consultants) took place between workshops. Greater 
interaction is required. 
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3. Few teams produced projects in depth. This is a limited weakness, since 
this was a secondary consideration and not the most important objective 
of the project. 

4. Successful interaction occurred between those vocational administrators 
and researchers that came together at the workshop. However, not enough 
teams came to the workshops* 

5. Larger (better) participation from more community colleges would have 

been nice. (19 of the 42 community colleges participated in at least 1 workshop) 

Rec ommenda t ions 

1. A process needs to be developed whereby more interim (between workshop) 
interaction and follow-* up takes place. This is seen as crucial to the 
production of greater integration in project development and research 
orientation. 

2. Follow-up a year from now is necessary to find out if the information 
that was made available and learned at the workshops is being used in 
any reasonable form by workshop participants. 

3. Perhaps a workshop series starting in the Spring and continuing through 

the Fall and into the next Spring would lead to more results (more Projects). 

4. A list of participants needs to be sent to all participants. A list of 
available consultants in all areas of expertise that were discussed at 
the workshop needs to be sent to all participants. A list of references 
for the alternatives and suggestions given at the workshops by the con- 
sultants to participants/ needs to be sent to ell participants. 

5. The Evaluation Report and the Auditor's report needs to be sent to all 
participants of the workshops. 

6. This auditor recommends continuance of the project or a modified project 
as a f ollow-up/continuatlon of this Integrated training/dissemination 

Q approach to research and administration in vocational education. 

ERIC 
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Generalized Summary and Conclusions 

The workshops were a moderate success. This Auditor sees the project as 
highly successful. 

The basic difficulty is: how to get the research man and the vocational 
education administrator in the conrounity college together with conisultants 
and produce an integrated leaming/research/actlon team or format* 

This project used what this Auditor would call the "rub of f "/action/motivation 
approach:, make information available, aid in determining where it is 
available, give supportive services/aid at any stage of development (plant 
the seed, give it a little waiter, wait for the sun to shine when required). 
People are motivated when they. need something and when these workshop 
participants need research techniques and aids, at the very least they now 
know where to go and they have the beginning of a background upon which to 
go from and grow. Two key points emerge from the project and lead this 
Auditor to state the project was successful: 

1. Vocational administrators and researchers are now more aware of having 

alternatives to choose from to satisfy their needs, 
2« The Project has planted seeds and given Information which can later be 

used when necessary or required for particular problems in the community 

colleges. 
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HOR CAL RESEARCH TRAINING WORKSHOP 
AGENDA 

Friday > November 3> 1972 

Session 

I 9*00 a.m> to 9:h^ a.m« Mr« Walter Brooks 

Orientation and Introduction, Coffee, Housekeeping Chores, 
Introductions, Overview, Objectives, Assignments 



II 9ihp a,m> to 10;U5 a.m> Dr> Ralph Smith 

"Research Concerns of Community College Vocational Administrators" 
"The Research Needs of Cbmmunity College Vocational Educators" 



III 10:t45 a>m> to I1;00 a,m> Break 



IV 11:00 a>m. to 12;00 (Noon) Dr« Kent G> Stephens 

"Organizational Structure, Behavior, and Communication Affecting 
the Research and Vocational Administrator" 
"Applied Educational Planning and Decision Process" 

V 12;00 (Noon) to 1:00 p>m. (Lunch) 

VI ItOO p>m. to 2:30 p,m> Dr> Kent G, Stephens 

"Systematic approach to Problem Solving" 

"Sophisticated vs. Appropriate operations research techniques" 
VII 2:30 p.m> to 2:^5 p«m. Break 

VIII . 3:00 p.m. to t»:30 p>m> Dr> Ben Gold 

Types of research, importance of design, researchable and non- 
reaearchable problems, significance sampling, experimental and 
Quasl'^ECTerimental designs^ external and internal threats to 
validity* 
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Saturday, November 4, 1972 

Session 

I 8;00 a,m, to 8:45 a.m, Dr, Ben Gold, Dr, Ralph Snlth, 

and Dr. Kent Stephens 

Summary and Review of Friday *s Sessions. 



II 8:45 a.m. to 10:15 a.m. L> Aughinbaugh, T* MacMillan, P> Preising 

Some specific aids in planning data collections use of census 

data (MacMillan) , tips on questionnaires and interviews (Aughinbaugh) , 

sources and kinds of data (Preising) . 

Ill 10;15 a.m> to 10:30 a.m. Break 



IV 10;30 a.m, to 12:00 (Noon) Paul Preising 

Division of participants into groups of 2-4 persons for discussion 
of concerns and projects (during this time the entire staff would 
"float around'* to these groups providing assistance according to 
their perceived needs, i.e. They may feel a need to pursue fur- 
ther topics presented earlier as they relate to their own concerns 
or projects). This will also allow for refinement of a short pre- 
sentation by the groups of their project or concern In the session 
following lunch. 



V 12:00 (Soon) to 1:00 p.m. Lunch 

VI 1:00 p.m. to 2; 30 p>m. Paul Preising 

Presentation by participants of their concern or project and how 
they propose to approach It. 



VII 2:30 jg^.m. to 3:00 p.m. 

Entire Staff (Panel with Walter Brooks as moderator) summary and 
where we go from here. 
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Evaluation Sheet 



Comments made by workshop participants - Workshop #1, November 3-4, 1972 
Friday 9 November 3 

A. "Research Concerns of Coiranunity College Vocational Administrators*' 

"The Research Needs of Community College Vocational Educators"- Dr. Ralph Smith 

1. Good general overview with specific examples, but he is not aware 

of the growth in sophistication of the current California Vo-Ed Dean, 

2. The need for G.E. Requirements at appropriate level. How do you 
get employers to give realistic level of competence needed. They 
all want college grads. 

3« We need to know more about our Vo-ed students if we ar^ to expect 
our faculty to understand them better. The Vo-Ed faculty, I 
believe, understands the nature of the student but the Gen. Ed. 
faculty probably doesn't. There isn't a cleavage in the faculty 
but the Gen. Ed. instructors are from a different world academically* 
We should be able to paint a picture pf the Vo-Ed student in terms 
of attitudes, aspirations, and abilities. How did the Vo-Ed 
faculty change attitudes of the academic faculty at Centralia in 
Washington? ' 

4. Good presentation for me, a new m.ember. Gave me a general overview 
to the problem. 

5. No more "off the top." 

6. How to use research to help Occ. Ed. 

?• As the "war" shuts down and carear ed. comes on, who will continue 
the battle for men^s minds? 

8. A very good introduction to the general problem of Voc. Ed. 

9. Good up-to-date review of the relationships that do exist in the 
community college in areas of career ed/research. Set a good 
tone for the workshop. 

10* Important in some respects but differ philosophically to academic 
and vocational criteria. What Is the purpose of vocational educa- 
tion and how is it to be determined? Agree with thoughts on 
teaching and research. 

11. Received valuable lnf0i» regarding concerns with which we should be 
familiar in evaluating programs. 

12. Research has not modified our programs* How can communicate with 
researchers to get the research we need that will let us sell pro- 
gram changes to the advisory committee and the community? 
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13. How do you prevent the vocational/academic schism? We are working 
on organizational plan and hope to prevent above. 

14. Need mimeographed material for distribution. 

15. Good general survey* Not anything particularly new, but good to 
be reminded about the things he spoke of. 



"Organizational Structure, Behavior, and Communication Affecting the 
Research and Vocational Administrator" - Applied Educational Planning 
and Decision Process" - Dr. Kent G. Stephens 

1. Excellent material — concise presentation, but perhaps too much, too 
quickly. A few ideas on taking a systematic approach to solve a 
problem and then more examples as to how to get started might have 
been enough for a starter. 

2. Excellent — System analysis and fault tree analysis as a special tool. 
I can use this on several ideas that are presently being looked at. 

3. Both presentations were informative. It leaves you feeling you 
need a lot more in systems. 

4. I saw no relationship between the exercise and the two topics listed 
here. The point that Dr. Stephens was trying to make was somewhat 
clarified through the questionnaire. 

5. Very informative on management techniques but somewhat questionable 
as to applicability to research techniques. Visuals looked im- 
pressive but Kent was frequently in the way and blocked the screen 
from view. 

6. Thought provoking - Presentation good. 

7. Very valuable information* A little too fast to comprehend details, 
but fine introduction to a method. 

8. As usual, he did a good jobi. I just nc^ed more experience with his 
system. HELPlI 

9. Specific mathematical formulae for research presentation was excellent. 
I would like more details on how researcher establishes priorities. 

10. Effective program on management analysis. 

11. Application of ideas to vocational education was very difficult — 
if there were any intended. 

12. Enjoyed this part. 

13. Good introduction. 

14. Too much material too fast. All good stuff but it would be much 
better to explore a few areas in depth, to get people involved . As 
it was, it was a "fast lecture" situation. 
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C. ''Systematic approach to Problero Solving" - "Sophisticated vs. 
Appropriate operations research techniques" - Dr* Kent Stephens 

1. The group came alive when they were participating — morfe ot. this 
is needed • 

2. Dr. Stephens is very well informed on sophisticated methods to 

be used for researching very difficult problems. i.His presentation 
covered too much too fast for non-research oriented people. A 
few examples of difficult research problems for Calif. Community 
College vo-Ed and research people would have made his presenta- 
tion more meaningful for me. 

3. Great presentation. I only wish that I had his presentation on 
tape • 

4. Enjoyed all that was done. 

5. Looks like an interesting but complex subject which will take 
some special training and workshops « 

6. This type of approach is great for large populations. Would like 
to know more about applicability to small groups. 

7. Techniques were good but their direct relationship to "our" purpose 
was always in my mind. 

8« Too much I too fast. Would have been completely lost if I had not 
been at Napa. 

9. Too much » too fast to be very useful, except as a very general over- 
view of what some people can do. Some good ideas (immediately useful) 
but these had to be caught "on the fly" while the rest of the dis- 
cussion went speeding by. (But — interesting.) Some of this 
material was covered (with much the same group) last year. 



D. Types of Research, importance of design, researchable and ^non-researchable 
problems, significance sampling, experimental and Quasi-Experimental 
designs, external and internal threats to validity - Dr. Ben Gold 

1* Excellent reference materials — like his slow easy pace and yet he 
covers a great deal of ground. Wonder if we were to take a typical 
vo-ed research problem and follow it through from beginning to end, 
if we wouldn't be of more immediate help to those attending? 

2. Excellent bibliography, general concepts, specific application. 

3. Lots of good ideas. 

4. Dr. Gold made a good presentation. It is apparent that he has had 
a lot of experience and can no doubt be of a great deal of help to 
individual vo-ed people and research people on individual problems. 

5. Good review of resources. 
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6. Glad to have the resource list. I will have to study the P-Picker 
approach more to gain its value* 

7. I appreciate getting the bibliography. Design is so far above me — 
it's too mucht Why don't you start with where I m and not where 
you think I ought to be? 

8. Seems to be the kind of information we need to work on research 
projects — good. 

9. Presentation on bibliography was helpful to me. 

10. Very valuable bibliography. 

11. Presentation of source material was great.' Handouts were of great 
value. The slides were not too interesting at this point. 

12. Excellent. 

13. Glad to have list of references. 

14. Tangible, specific, adequately paced (not too fast), clear, 
operationally useful and helpful. People became involveti. 



Saturday, November 4 

A. Summary and review of Friday's sessions. Dr. Ben Gold, Dr. Ralph Smith, 
and Dr. Kent Stephens 

1. Stephens did a great job. Gold and Smith didn't contribute much 
that I found relevant. 

2. Gold's bibliography will be helpful in the future. The Pea-Picker 
material offers me little. 

3. Gold - Solid, good ideas on book of research 

4. Smith - Not effective. 

5. Stephens - Excellent, maybe a little too much for vo-ed people. 

6. OK - I did not hear all of this. 



. Bl. Tips on Questionnaires and Interviews - Lorine Aughinbaugh 

1. Both Lorine and Tom MacMillan have done a good job in bringing in 
relevant information. Let's have Lorine 's comments duplicated. 
Your next effort should be less talking £t and more talking with . 

2. Solid - many useful specific points. 

ERIC 



3. Great - Could we get a copy of these very concise notes? 



4. OK - Much came out in our group discussion on Follow Up studies. 
Her summary of her notes was good - but too late in the morning 
for people to be eager to take notes. 

Use of Census Data - Dr. Tom MacMillan 

1. Did a good job in bringing in relevant information. 

2. Excellent resources and methods of utilization of hard data already 
available to researchers as a basis for verification or point of 
departure for identification of problem. 

3. I am writing for mine ASAP (as soon as possible). 

A. This review session was the best part of the conference. 
5., His report to the whole group at 11 was OK, but brief. 

Refinement of a short presentation by the groups of their project or 
concern. 

1. This session has been helpful. It put the brakes on what I was 
doing. I am now going to ask some questions and go back to the 
beginning. I like this better than yesterday's session. 



ATTENDANCE LIST 
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Name of Participant 



Title 



College or Organization 



Lorine Aughinbaugl^ 
Dean O'Brien 
Ralph B. Smith 
Kent Stephens 
Jim Mitchell 
Earl P. McGhee 
Lance Rogers 
Bob Deal 
Kenneth Jackson 
Vincent P. Pad ilia 
Henry Walters 
Bill Keehn 
Florin Caldwell 
Bill Preston 
Charles Peterson 
Bruce Palmer 
Gerald Fries 
Gary L. Graham 
Ben Gold 
Dayton Axtell 
Harry Shortess 
Ron Leonard 
Ralph Thompson 
Sidney McGaw 
Paul Preising 
Tom MacMillan 
Ray Rosales 
Walter Brooks 
Bob Nichols 
Dean Eaton 
Fred Dagenais 



Assistant Dean of Research 
Electronics Instructor 
Chairman-Dept. Ed* Adm. 
Educa. Admin. Department 
Director-Transfer Ed. 
Director-Career Education 
Director-Tutorial & COIL 
Assoc. Dean of Instruction 
Instructor 

Assoc. Dean/Even. & Summer 

Career Planner /Research 

Research Analyst 

Director of Research 

Director of Research 

Instructor 

Research Assistant 

Coord. Inst. Research 

Coord . Handicap Prog , /Coun • 

Director of Research 

Psychometrist 

Assist. Dean - Voc. Ed. 

Counselor 

Research 

Director -Voc. Ed, 
Director of Research 
Research 

Vo. Ed. Counselor 
Director of Research 
Dean of Guidance/Admissions 
Director of jAdmissions 
Consultant 



American River College 
American River College 

Brigham Young University 

I 

Brigham Young University 
Butte College 
Butte College 

City College-San Francisco 
Columbia Junior College 
Contra Costa College 
Cosumnes River College 
Cosumnes River College 
De Anza College 
De Anza College 
Diablo Valley College 
Feather River College 
Foothill College 
Fresno City College 
Fresno City College 
Los Angeles City College 
Merritt College 
Merritt College 
Monterey Peninsula College 
Sacramento County Office 
San Jose City College 
San Jose City College 
Santa Barbara C. College 
Santa Barbara C. College 
Shasta College 
Shasta College 
Solano College 
U. C. Berkeley 
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NOR CAL WORKSHOP 
November 3-4, 1972 
Ben K. Gold 

I. Planning a Research Study 

"Research Is a way of dealing with Ideas. It Is nothing more than thls» 
and It Is nothing less" Barnes 

A. Characteristics of Researchable Problems 



1. 


Important 


2. 


Solvable in principle 


3. 


Answers not already available 


4. 


Reason alone is not enough reason 


5. 


Manageable size 


6. 


Concretely stated 


1. 


Measures available 


8. 


Favorable situational considerations 


Types of Research (Isaac and Michael) 


1. 


Historical 


2. 


Descriptive 


3. 


Developmental 


4. 


Case and field 


5. 


Correlational 


6. 


Causal Comparative or "ex post facto 


7. 


True experimental 


8. 


Quasi-experimental 


9. 


Action 



Barnes says two types: Historical-looking from present to past 

Experimental-looking from present to future 

Beaird (in CORD Manual) says you do one or more of three things: 
describe, compare, relate 

C. Questions worth the J. C. Institutional Researcher^s time: (Hoyt)* 
(three "layers" of research) 

1. What characfi^ristics that are important to education does 
a given student or group possess? (describe) 

2. How does one group differ from one another? (compare) 

3. What works and under what conditions? (relate) 

D» Steps in the Research Process (Barnes) 

1. Selecting the problem 

2. Accumulating pertinent knowledge and Information 

3. Latent Period 

4. Idea or hypothesis formation 

5. Designing the test of the hypotheses 

6. Critical analysis and evaluation of observations made 

7. Rejection or acceptance 



*Donald Hoyt — presentation to 1967 CJCA Research Conference "The Preparation 
and Effective Use of Research Models at Your College" 



(Isaac & Michael) 

1. Identify the problem area 

2. Survey the literature related to it 

3. Define the actual problem for investigation in clear, 
specific terms 

4. Formulate testable hypotheses and define the basic 
concepts and variables 

5. State the underlying assumptions which govern the inter- 
pretation of results 

6. Construct the research desigii to maximize internal and 
external validity 

7. Specify the data collection procedures 

8. Select the data analysis methodology 

9. Execute the research plan 

10. Evaluate the results and draw conclusions 

E. Hypothesis Formation 

1. Hypotheses - various possible explanations for the condition 
or event 

differ from assumptions - assumed truths 
differ from theories - broader, more general 

2. Result from observing, collecting facts, surveying literature, 
mulling over, and calculated guesses 

3. Criteria for evaluating hypotheses 

a. Plausibility 

b. Testability 

clear and precise 

c. Adequacy of scope 

does it explain all (or at least most) of the facts? 
d* Compatibility with present knowledge 

4. Important as base for research design 

5. Research (positive) hypothesis vs null (negative) hypothesis 

F. The Research Design - the plan for the project 

1. Questions to ask 

What approach is best? 
Are pertinent data available? 
What variables are relevjant? 
How will data be collected? 
How will subjects be selected? 

What instruments will be used? (select or develop) 

Is pilot study desirable? 

Is there need to train data collectors? 

What analysis techniques are available? 

Will computer be available and/or helpful? 
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2. Experimental design 

a. Design is of critical importance in experimental studies 
Two considerations 

(1) What questions will the design answer? 
(and not answer I) 

(2) What is the relative information gain/cost 
picture? 

c. Design entails: (Crawford in CORD Manual) 

(1) Selecting or assigning subjects to groups on 
experimental units 

(2) Selecting or assigning units for specific 
treattflGnte or conditions of the experiment 

(3) Specifying the order or arrangement of the 
treatment or treatments 

(4) Specifying the sequence of observations to 
be taken 

(does not include: details of sampling, selec- 
tion of measurement instruments, selection of 
research problem, nuts and bolts of procedures, etc.) 

d- "Max-min-con*' (Ker linger) 

Design should: 

(1) Maximize effects due to experimental variables 

(2) Minimize effects of chance variables 

(3) Control the effects of extraneous or unwanted variables 
e. Validity 

(1) Internal - does the treatment make the difference? 

(2) External - can you generalize the results? 

f» Threats to validity (Campbell & Stanley) 

(1) Internal 

History 

Maturation 

Testing 

Instrumentation 
Statistical regression 
Differential selection 
Experimental mortality 
Interaction affects 

(2) External 

Reactive effect of testing 

Interaction effects of selection biases and 

the experimental variable 
Reactive effect of experimental arrangements 
Multiple-treatment interference 
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3. Sampling 

a. All research studies involve sampling — subjects, 
behavior, test questions 

b. Population: sample = total: part - parameter: statistic 

c. Types of sampling 

(1) Whatever is available 

(2) Simple random 

(3) Stratified 

(4) Cluster 

d. Generalization from sample to population requires 
random sampling 

e. Random number tables 

f • Statistical inference statements about populations 
based on random samples 

Two types 

(1) Estimation of parameters 

(2) Tests of hypothesis (significance) 

II. Data Collection and Analysis 
Next workshop I 
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RECOMMENDED FOR THE J, C, RESEARCHER^ S LIBRARY 
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* BARNES, Fred P., Research for the Practitioner in Education ^ Dept. 

of Elementary School Principals, N.E.A., 1964. 

Although written primarily for elementary school educators, the 
material is general enough to be of help to the junior college 
researcher. An excellent introduction to concepts of research 
design and analysis* Easy to read. 

* CAMPBELL, Donald T., and Stanley, Julian C, Experimental and Quasi- 

Experimental Designs for Research , Rand-Mr.Nally & Co., 1966 

This book was originally a chapter in the Handbook of Research 
on Teaching, edited by N. L. Gage. It minimizes statistics, 
emphasizes threats to validity of each design. An important book, 
thorough and clear. 

* FREUND, John E. and Williams, Frank J., Dictionary Outline of Basic 

Statistics , McGraw-Hill, 1966. 

A dictionary of practically everything in statistics, followed by 
a section of statistical formulas. Not a taxt book, but a handy 
reference to have around. 

* HUFF, Darrell, How to Lie with Statistics , W. W. Norton & Co., 1954. 

A humorous little book with a powerful message about misuses of 
statistics. Should be read by everyone who presents or interprets 
statistics. 

* ISAAC, Stephen and Michael, Wm. B., Handbook in Research and Evaluation , 

Robert R. Knapp, Publisher, San Diego, 1971. 

"An overview, a summary of alternatives, an exhibit of models, 

a history of strengths and weaknesses." An excellent handbook for 
the "occasional" researcher. 

KERLINGER, Fred N., Foundations of Behavioral Research , Holt-Rinehart 
Winston, 1965. 

Lots of information in this book. Its 739 pages include something 
about anything related to scientific approaches to research,. Lucidly 
written, a minimum of technical jargon. 

OPPENHEIM, A. N., Questionnaire Design and Attitude Measurement, Basic 
Books, 1966. 

If you are planning a survey using a questionnaire, interviews, or 
homemade attitude scales, it will pay you to consult this book first. 
Lots of suggestions, probable pitfalls, etc. 

* paperbook 
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Recommended for the J. C. Researcher's Library (continued) 

* SAX, Gilbert, The Construction and Analysis of Educational and Psycho- 

logical Tests , College Printing and Typing, Inc., Madison, 
Wisconsin, Revised 1968. 

A workbook describing how to construct, analyze, and interpret 
tests, both standardized and teacher-made. Assume no back- 
ground. Goad sections on reliability and item analysis. 

SHAW, Marvin E. and Wright, Jack M. , Scales for the Measurement 
of Attitudes , McGraw-Hill, 1967. 

An excellent source book for existing instruments for measuring 
a variety of attitudes. Scales are described and critiqued, with 
sample items from the many instruments included. Highly recom- 
mended to have available. 

SIEGEL, Sidney, Nonparametic Statistics for the Behavioral Services , 
McGraw-Hill, 1956. 

A "must" book. Details in clear language with examples of 27 
nonparametic tests* Also a good discussion of the rationale 
behind them. 

* SMITH, G. Milton, A Simplified Guide to Statistics for Psychology 

and Education , Ath edition, Holt, Rinehart and Winston, 1970. 



STUART, Alan, Basic Ideas of Scientific Sampling , Hafner, Publ. Co., 
1962. 

A brief book (98 pp.)> written for the "non-specialist," indicates 
advantages and disadvantages of various types of sampling — simple 
random, stratified, cluster, etc. 

WALKER, Helen M. & Lev, Joseph, Statistical Inferences , Henry Holt & 
Co. (revised edition), 1958 

There are many good statistics tests available. This is one of the 
better ones, from the standpoint of usefulness to the junior college 
researcher. 

* WEBB, Eugene J. > etal. Unobtrusive Measures: Nonreactive Resear<:h^in 

the Social Sciences , Rand-McNal ly , 1966. \ ^""^^-^^ 



This book details some, interesting and useful methods of obtaining 
data using "measures that do not require the cooperation of a re- 
spondent and that do not themselves contaminate' the response." 



A useful set of materials to have on campus (for $15) is the CORD National 
Research Training Manual and Workbook. , It can be obtained from Teaching Re- 
^ search, Monmouth, Oregon 97361. 



An excellent small book explaining in layman's language some 
of the more important concepts in statistical analysis* Es- 
pecially good on significance tests and analysis of variance. 
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Recommended References Useful in Analysing 1970 Census Data 



1* Higher Education Measuronent and Evaluation Kit 

Center for the Study of Evaluation 
Graduate School of Education 
University of California 
Los Angeles > CA 



2, California Statistical Abstract 

State of California. 
Documents Section 
P. 0. Box 20191 
Sacramento, CA 95820 



3. i970 Census Population-General Population Characteristics 
Final Report PC(1) - B6 

U.S. Dept» of Commerce 
Bureau of the Census 
Washington, D. C. 

4. 1970 Census of Housing - HC(1) - A6 

U.S. Dept of Commerce 
Bureau of the Census 
Washington, D. C. 

5. 1970 Census ol' Population and Housing ~ PHC(l) 

U»S. Dept. of Commerce 
Bureau of the Census 
Washington, D» C. 



6. Forecasting Occupational Opportunities: Quantitative Procedures 
and a Case Study of Santa Barbara County 

General Research Corporation 

P. 0. Box 3587 

Santa Barbara, CA 93105 
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NOR CAL WORKSHOP 
November 3-4, 1972 
Lorine A. Aughinbaugh 

Tips on questionnaires and Interviews. 

Primary Source Data 

Interview Method 
Advantages 

1. A higher degree of accuracy Is attained through the acquisition of 
material direct from the source. 

2. Material is often obtained that cannot be secured through the ques- 
tionnaire. 

3. There is opportunity personally to check information acquired. 

.> 

Disadvantages 

1, Only small samples can be gathered • 

2, The subjective factor is involved in recording by interview. 

3, The method is generally inefficient, and the time and expense involved 
necessarily mean limited field coverage. 
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Questionnaire Method 
Characteristics 

1, The questions should be easily understood. 



2. I£ possible they should be arranged in logical sequence. 



3* The answers should consist of yes or no, check or blank space, or 
numerical indication where possible. 



4. The questionnaire should be concise. 

5. It should be in the most convenient, answerable form. 

6. It should be constructed so as to icilitate the tabulation of data. 
Advantages 

1. A large area may be easily and quickly covered. 



2. The method of assembling data is relatively inexpensive. 
Disadvantages 

1. Frequently questions cannot be answered without a supplementary explanation. 

2. In many cases the results are unreliable. 

3. A large part of the sample taken may not answer the questionnaire. 



From: Arkin and Colton - Statistical Methods, 4th edition, c 1961 - page 150. 
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Second Research Training Workshop 
for Vocational Educators 



December 7-8, 1972 
SUMMARY 

Workshop Agenda 

Evaluation.>y Participants 

List of Participants 
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MEMORANDUM 

TO: Nor Cal Research Training Workshop Participants 

FROM: Walter L. Brooke 



DATE: November 27, 1972 

SUBJECT: Second Research Training Workshop 

The second of the three Nor Cal researlch training workshops will be 
held at American River College on Thursday 'evening, December 7, from 7:30 
to 10 p.m., and Friday, December 8, 8:30 a.m. through 3:30 p.m, Lorine Aughin 
baugh is sendlnp vou parking permits and maps. The tentative agenda is shown 
below: 

Thursday J December 7, 7:30 to 10 p.m. ~ Review 

On Thursday evening > we will review progress made during the first 
workshop. A summary of the first workshop proceedings including copies 
of hand out materials will be available on Thursday evening. Those of 
you who were unable to attend the first workshop should be sure to be 
there on Thursday evening. 

Friday, December 8» 8;30 to 10:30 a.m. - Critique of Projects 

On Friday morning, participants will be asked to describe the re- 
search project they have undertaken. Duringv^he finai session of the 
last workshop, it was found that projects tendted to fall into two general 
classifications. About half the participants vi\l be developing an occu- 
pational needs assessment plan for their college while the remainder will ; 
be developing a plan to follow up students in occupa^onal training pro* 
grams. Kent Stephens has asked that we answer two questions about our re- 
search projects as a first step in understanding svstMiatlc problem solving. 
The answers to these two questions correspond roughly to the first step in 
system analysis as described by Dr. Stephens. Participants should come to 
the Friday morning session prepared to answer the following questions about 
their project: 

1. Why do you want to complete the project? In answering this ques^ 
tion, be as specific as possible. Try to list all the reasons why 
the project ought to be completed. Is the information- necessary 
to the operation of the college? In what way is it necessary? 
What functions will be improved as a result of this project? 

2. Who will be affected by what you are doing? In answering this 
questioa» you should be specific In describing what the effect 
will be on significant members of the college community. 

Friday, December 8^ 10;45 to 12:00 a.m. - Problem Identification & Analysis 

Kent Stephens will make a formal presentation in the morning on prob- 
lem identification and analysis, the third step in his systematic problem 
solving model. We will go further with a problem solving model if it ap- 
pears there is time for it. 
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Friday, December 8, 1:00 to 3:30 p.m, - Research Project Clinic 

On Friday afternoon, we are going to try something a little different 
which you may find a useful learning experience. Each of the workshop 
consultants will be available at six different tables to give specific as-* 
slstance to participants on their project or to discuss the following top- 
ics with you In general terms: 

Problem definition - Dr. Kent Stephens 

Experimental design - Dr. Paul Preising 

Census data analysis - Dr. Tom MacMlllan 

Questionnaire and interviews - Lorlne Aughinbaugh 

Statistics and sampling - Dr* Ben Gold 

Follow up - Dr. Ralph Smith 

See you next Thursday evening. 
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Evaluation Sheet 

CoHimehts made by workshop participants . - Workshop #2, December 7-8, 1972 
Thursday evening, December 7 

A. Needs Assessment Group - Lorine Aughinbaugh 

1. Interesting and helpful. 

2. We had a good discussion, but not certain that as much specific 
information was given as might have been desirable. 

B. Follow Up Group - Dr. Tom MacMillan 

1. I felt this was good. It raised a question about an 2:.ssumption I 
had made. This kicking around is good* Only problem is that 
sometimes it becomes dominated by one person. No fault of the 
leader. 

2- Quite helpful. 

C. Catch Up Session - Dr. Kent Stephens, Dr. Ben Gold, & Dr. Ralph Smith 

1. I feel that this was a valuable session in that the review brought 
me to a point where I was able better to understand what was going 
on Friday. 

2, Too brief, but excellent. 



Friday morning, December 8 

A. Individual Project Reports 

1. I learned that almost everyone has essentially the same problem 
relative to research needs. 

2. Helpful^ 

3. Stimulated some new ideas for our ownuse. 

4. Gave me an idea of who to contact about a survey. 

5. Good 

6. Good to see what other schools are planning. 
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Friday morning^ December 8 (contQ 

B.. Occupational Information - Mr. Ralph Thompson 

1. Helpful 

2. I (we) feel strongly that Mr. Thompson's program for Job market 
analysis, if as accurate as he claims, should be pushed by 

Nor Cal to CJCA and the Chancellor's Office for state-wide im- 
plementation. This should be an essential part of every 
community college's data base for voc. ed. decision making. 

3. 'This is a dimension we should explore further - Nor Cal meeting^ 



C. Smorgasbord with Dr. Ben G o ld, Dr. Ralph Smithy Dr. Paul Preising, 
Dr. Kent Stephens, and Mrs* Lorine Aughinbaugh . 

1. Probably the most efficient way to deal with specialized individual 
problems. 

2. Very good. I prefer structured presentations. 

3. Most of this session was spent with Dr. Stephens — very helpful in 
defining our particular problems, 

4. Good. It helped clarify a question I had in my mind about asking 
questions. 

5. Good. 

6. This seemed to satisfy several needs. 




.( ^0 

ATTENDANCE LIST 
Workshop ^2 - December 7 & 8, 1972 



Name of Participant 


Title 


College or Organization 


Lorlne Aughinbaugh 


Assistant Dean of Research 


American River College 


Ernie Droege 


Division Chairman T V 


American River College 


Louis Quint 


Assist. Dean, Voc. Ed. 


American River College 


Ralph Smith 


Chairman-Dept • Ed . Adm* 


Brigham Young University 


Kent Stephens 


Educa. Admin. Department 


Brigham Young University 


Jim Mitchell 


Director— Transfer Ed . 


Butte College 


Earl McGhee 


Director-Career Education 


Butte College 


Lance Rogers 


Director-Tutorial & COIL 


City College-San Francisco 


Gary M» Jerome 


Consultant 


Chancellor's Office 


Robert Deal 


Assoc* Dean of Instruction 


Columbia Junior College 


Jon M. Hagstrom 


Assoc* Dean of Instruction 


Columbia Junior College 


Phyllis Goldman 


Director of Testing 


Contra Costa College 


Kenneth Jackson 


Instructor 


Contra Costa College 


Vincent Padilla 


Assoc Dean/Even. & Summer 


Cosumnes River College 


Henry Walters 


Career Planner/Research 


Cosumnes River College 


Charles Peterson 


Instructor 


Feather River College 


Gerald Fries 


Coords Inst. Research 


Fresno City College 


Richard Handiey 


Dean of Vocational Education 


Fresno City College 


Ben Gold 


Director of Research 


Los Angeles City College 


Dayton Axtell 


Psychometrist 


Merritt College 


Harry Shortess 


Assist. Dean — Voc# Ed. 


Merritt College 


Virginia Murdoff 


Dean of Students- 


Napa College 


Ralph Thompson 


Research 


Sacramento County Office 


Joe Leal 


Coord, of Industrial Education 


San Jose City College 


Paul PrPlQino 




doll uUoCl t^XLjr ^LlXX^gC 


Tom MacMillan 


Research 


Santa Barbara C. College 


Ray Rosales 


Counselor 


Santa Barbara C. College 


Joyce Welch 


COIL Director 


Santa Rosa Jr. College 


Walter Brooks 


Director of Research 


Shasta College 


Lloyd Livingston 


Dean of Vocational Education 


Shasta College 


Martin Taylor 


Dean-Student Pers* Services 


Sierra College 


Dean Eaton 


Director of Admissions 


Solano College 
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Third Research Training Workshop 
for Vocational Educators 



March 23-24, 1973 
SUMMARY 

Workshop Agenda 

List of Participants 

System Analysis of Occupational Student Follow Up . . 

Solution Alternatives Selected by Workshop 

Participants 



MEMORANDUM 



TO: Nor Cal Research Training Workshop Participants 

FROM: Walter Brooks, Chairman 

DATE: March 14, 1973 

SUBJECT: Research Training Workshop 



I hope you all remember that we have scheduled a research train- 
ing workshop for Solano College the evening of March 22 and all day 
March 23. We will meet in the Boardroom in the Administration Building 
on the 22nd at 7:30 p.m. and then again in the Boardroom at 9:00 on 
Friday morning. Here is ths agenda: 



AGENDA 



Thursday, 

7:30 - 10 p.m. 



Development of a model occupational follow-up 
system : 

In this session, I am going to ask several col- 
lege representatives to bring samples of their 
current follow-up procedures and instruments. 
What we will begin to do as a group is to de- 
velop a follow-up system incorporating^ the best 
of what each of us are doing. The idea is to 
begin to build a practical and feasible model 
which would meet the needs of each of our dis- 
tricts. 



Friday, 

8:30 - 9:00 a.m. 



Coffee and donuts. 



9:00 - 10:00 a.m. Report on workshop projects : 



In this session, workshop participants who have 
developed a project as an outgrowth of their 
participation in the workshop will have an oppor- 
tunity to report on the progress of the project. 
From the response I have received thus far, not 
many colleges have gotten particularly elaborate 
with the research project. It would probably be 
unrealistic to expect anything different from a 
group of busy people but this will be an oppor- 
tunity for those who have undertaken a project to 
describe what they have done. I would hope we could 
keep this informal as we did the last time. I will 
be contacting those people who returned the form 
indicating they had undertaken a project and allow 
you as much or as little time as you need to dis- 
cuss what you have been doing. 



Friday, 

10:30 - 12:00 a.m. 



Description and discussion of a model occupational 
follow-up system : 



During this session, we will continue our work on 
developing a model follow-up system. At this time, 
we will review the progress from the night before 
and attempt to agree upon a final description of 
an effective and feasible follow-up system. 



12:00 - 1:00 p.m. Lunch 



1:00 - 2:00 p.m. Open discussion on research techniques and method- 
ology : 

I am confident in my own mind that a good deal of 
learning has taken place in these workshops which 
will be useful to the participants in the future 
in planning, revising, or evaluating occupational 
programs. I think that a review session giving 
participants an opportunity to discuss concepts 
introduced during the workshop would be helpful. 



2:00 - 3:00 p.m. Discussion of a subsequent research training work- 
shop ; 

If sufficient interest exists, we might consider 
conducting one more research training workshop in 
v^ich we would develop a needs assessment model 
similar to the follow-up''model we will have coop- 
eratively developed. Scheduling a subsequent work- 
shop will depend upon interest of participants and 
success of our efforts in developing a model follow- 
up system. 



Those of us who will be traveling a long distance to the workshop 
plan to stay at the Holiday Inn in Fairfield Thursday evening, although 
you are certainly free to make your own arrangements. As in previous 
workshops, the expenses of participants will be paid. 



See you at Solano College* 
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System Analysis of Occupational Student Follow Up 



Needs Assessment 



'What is 



'What should be' 



tl 



1. 



Community college vocational educators 
currently do not have reliable informa- 
tion on the performance of former stu- 
dents on the Job. 



1. 



Vocational educators should know 
the impact of their training pro- 
grams on the student for realistic 
educational planning. 



2. Occupational educators cannot make 
accurate reports to the state regard- 
ing students in occupational programs. 
Currently, vocational educators are 
forced to make estimates. 



Mission Objective r • 

By 5 p.m. on Karch 23, the research training workshop participants will outline a 
procedure which can be refined on each college campus to follow up occupational stu- 
dents and provide information which will satisfy both reporting needs and system 
planning needs of occupational educators* 

Performance Requirements 
Expected Outcomes: 

The follow-up system developed during the research training workshop should: 

1. Be descriptive of the total occupational student population. 

A. Should satisfy the descriptive needs of the vocational dean, 

the counselor, vocational advisory committee, and state reporting. 
(1) Description should include sex, age, race, disadvantaged 
status, handicap status » Job identification, salary range 

2. Serve as a guide to the educator in modifying occupational program 
characteristics* 

A* Provide infomation on placement ratios, needs for curriculum re- 
vision, match and miss-match between training and industrial need. 

3* Should provide information on all occupational programs offered. 

4. Should be systematic and on-going • 

5. Should provide for employer evaluation of occupational program ob- 




jectives. 



6* Should describe why a student is not working in the area for which he 
is trained. 



7. Should provide for short-term and long-term follow up. 
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Necessairy Constraints 
!• The system must be inexpensive. It must require no more than 



professional staff time and clerical time for its completion each 

semester. The materials and supplies required should cost no more than 



2. The system must be sufficiently adaptable and simple to implement that it can 
be adopted without outside consultation on each community college campus of 
workshop participants. 

3. The sj'Stem must lend itself to data processing but machine processing should 
not be a necessary requirement of the system. 

4. The systemjiiust be capable of identifying and developing information on 80% 
xfcupational student population. 




Definitions 



For the purposes of this system, students will be classified in four ways: 

1. The Undetermined General 

Students classified as "undetermined general" are students v^ho have no course 
in their student study list which is required in the curriculum of any occupa- 
tional program. An example of this would be the transfer student taking all 
lower division general education courses which relate to a state college or 
university program. 

2. Undetermined Vocational 

Students classified as "undetermined vocational" have at least one course in 
their student study list which is required in an occupational curriculum but 
the course is of a general nature and cannot be specified as belonging to a 
particular occupational area. An example of this would be the student with 
a Technical Math course In his study list which was required in several occu- 
pational majors. 

3. Determined Vocational Area 

Students classified as "determined vocational area" are those who are taking 
at least one occupational course which can be identified with an occupational 
subject area but is not sufficiently specific to describe as belonging to one 
occupational major. An example would be a student taking a Soils class In the 
Ag. department which was required only in the Agriculture-Natural Resources 
area and not in other vocational subject areas. . 

4. Determined Vocational Major 

Students classified as "determined vocational major" Include one or more 
courses in their study list which are of a sufficiently specific nature to 
be described as relating to one occupational major. An example of this is a 
student who had Introduction to Law Enforcement and Criminal Law on his study 
list which relate only to a Police Science major. 
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Solution Alternatives Selected by Workshop Participants 
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Model Follow-Up System . 



I. Ask all instructors teaching courses designated as vocational education 
to read the memorandum requesting information on students' future occu- 
pational plans. 



PRE-FOLLOW UP MEMORANDUM 



TO: All instructors in vocational subject areas 

FROM: Vocational Education Dean 

DATE: Three weeks prior to final exams 

SUBJECT: Student Occupational Placement Information \ 



We are now in the process of preparing for the follov-up study in 
occupational majors. It is very important that we get the names of stu- 
dents who plan to seek work in an occupational field in which they have 

trained here at College • Would you please read the 

following message to students in classes listed at the bottom of the 
page. 

"How many students In this class plan to seek full time employment 
at the end of the semester (or quarter) rather than return to college? 
All students who have not yet done so, should fill out a Student Occupa- 
tional Placement Information form. This information is needed to assist 
you in your search for a job and to maintain contact with you when your 
formal training is completed to evaluate our instructional program. It 
is not necessary for students to supply this information if they have al- 
ready done so in another class. Students who decide later to discontinue 
their education and seek work should drop by| the' Placement Office and 
fill out one of these forms when the declsipn is made.^* 

Please collect all the forms and return to the Office of the Voca- 
tional Education Dean by the end of the week. 



The forms should be passed out in the following classes: 
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Instructor 



Course 

STUDENT OCCUPATIONAL PLACEMENT INFORMATION 

Students planning to seek full time employment at the completion of this semester 
in a Job related to the course work they have taken at College should complete 

the following fonn* 

NAME ^COLLEGE MAJOR 

PERMANENT ADDRESS 

Street City State 

Job or occupational area in which you intend to seek employment; 



Please check the appropriate statement: 

I am fairly certain I will be employed full time with (name and address of em- 

ployer) ^ ^— 

Job Title: _^ . 

^I think I may have a job but am not sure. (Name and address of prospective em- 

ployer) 

. Job Title ■ . 

I have no prospects for employment at this time and I would appreciate placement 

assistance. 



A. Students who indicate they are certain of employment would then be 
asked to complete the student follow up questionnaire (see page 49). 

Uhen the in-class follow up questionnaire has been completed by students: 

A. Develop 3 lists from the returned questionnaires to indicate students 
who : 

1* Are certain of employment 

2. Are not sure of employment 

3, Have no prospect of employment 

B. Refer names of students indicating "Not sure and. No prospect" to the 
Placement Office and division chairmen. 

C. Refer total list back to division chairmen for review and addition 
of missed students. 

Three weeks after the beginning of the new semester: 

A. Check the students identified for follow up against the current en- 
rollment roster and eliminate students who returned to college. 

B. Determine the cost of following up all students identified. If the cost 
appears prohibitive, select a random sample of students for contact. 

C. Send students the follow up questionnaire shown on the following page 
with an appropriate cover letter. 



II. 



III. 
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Occupational Student Follow-Up Questionnaire 
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Name \ Present Address^^ 

Telephone Numbe r Social Security Number^ 

What was your first job after leaving college? 

Who was your first employer ? 

FLEASE ANSWEP THE FOUOWING QUESTIONS RELATING TO YOUR PRESENT JOB. 

What is your present job ? 

Who is your present employef?_^ . 

Do you work full time ? Or part time ? 

Do you work day shift? Swing shift? Graveyard? Other? 

How many hours per week do you work ? ^ 

What is your hourly rate of pay ? What is your monthly rate of pay? 

How long have you held this job?^ ■ 

How has your college training helped you in your present jobi to get the job ? 

to earn a promotion ? other ?^ 

(please explain) 

Were you given help by the Placement Office or the instructional st^f of the college in seeking 
your first job? 

Yes No Please explain. 

Are you satisfied with your present employment? 

Ye s N o Please explain. 

We would like to have you rate various aspects of your college experience as it relates to your 
present work. (Please check the appropriate column*) 

Ex eg 1 lent Good A cceptab le Poor 

a« College instructional program 
b« Training facilities & equipment 

c* College work experience program 

d. College counseling services ! 

Which course or courses taken while in coneg.e have been most helpful to you in your work? 
Explain > 



What additional training or skills would have been most uieful for you to develop? Explain^ 



Comments: Please provide any information about yourself^ or the college, which you feel would 
be helpful to us in improving our occupational programs* . 



This questionnaire is designed to be printed on card stock and folded in the 
center. A stamp and the address of the college is placed on the outside of 
the card. The card is included with the follow up letter to the student. 
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IVr Send a second and third follow up letter and questionnaire to students 
not responding to the first at two week intervals. 

V, From the follow up returns, identify a sample of Jobs which typify 
the target jobs In a specified vocational major. After a six month 
interval, interview employers identified through this process to re- 
late job training to job skills of the student. 

VI • Prepare a follow-up report 

A* Match student returns with the permanent student record to 

relate descriptive data to the follow up. This should include 
sex, age, race, disadvantaged status, handicap status, college 
major, and vocational education courses taken. 

Classify students. under one of the four classifications de- 
veloped for this system: undetermined general, undetermined 
vocational, determined vocational area, oridetermined voca- 
tional major. Report the percentage and number of students 
in each category. 

C* In the report, identify students working in the area for which 
they are trained and those not working in the area of training 
by major or area of concentration*. 

D. Report on statements of courses students found most helpful. 

E. Report on training or skills most useful to develop. ^ 

F. Report on amount and kind of help received in getting first job. 

G. Report on student evaluation of instructional program, training 
facilities and equipment, work experience program, and counsell- 
ing services. 

H* Report on the relationship of GPA of student^; in occupational 
courses to successful placement and job satisfaction. 

. - \ 

VII. Dissemination of the report 

A» Division chairmen 

B. Occupational advisory committee 

C* College administrative cabinet 

D. Board of Trustees UNIVERSITY OF CALIF 

LOS ANGELES 
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